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Efficient Rate Allocation for Multi-resolution 
Coding of Data 

Field of the Invention 

This invention relates generally to digital signal compression, coding and 
representation, and more particularly to an image compression, coding and 
representation system using rate control or rate allocation and having both 
device and method aspects. It further relates to a computer program product, 
such as a recording medium, carrying program instructions readable by a 
computing device to cause the computing device to carry out a method 
according to the invention. 

Background of the invention and prior art 

Due to the huge size of the raw data of digital signals, compression must be 
applied to the raw signals so that they may be transmitted and stored. The 
digital signals can be video, image, graphics, audio, speech, etc. In particular, 
digital image signals can be very large in size. Digital cameras can be used to 
capture high resolution images that can easily have a resolution of 10 mega- 
pixels or higher. Many remote sensing images or map images can have very 
high resolution as well. Image compression is very important for the storage, 
transmission and representation of such digital images. 

One important international standard for image compression is the ISO/IEC 
10918 standard, known commonly as the JPEG (Joint Photographic Experts 
Group) standard [G. K. Wallace, "The JPEG Still Picture Compression 
Standard," IEEE Trans. On Consumer Electronics, vol. 38, no. 1, Feb. 1992.]. 
The JPEG standard was finished in early 1990s and has since been used 
widely in internet and digital cameras. In the year 2000, ISO/IEC produced a 
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new standard ISO/IEC 15444, known commonly as the JPEG2000 standard 
[ISO/IEC, ISO/IEC 15444-1: Information technology— JPEG 2000 image 
coding system — Part 1: Core coding system, 2000: ISO/IEC, ISO/IEC 15444- 
2: Information technology— JPEG 2000 image coding system — Part 2: 
5 Extensions, 2000:C. Christopoulos, et. al., "The JPEG2000 Still Image Coding 
System: An Overview," IEEE Trans, on Consumer Electronics, vol. 46, no. 4, 
Nov. 2000.], which can give both objective and subjective image quality 
superior to JPEG. 

Baseline JPEG uses mainly discrete cosine transform (DCT), scalar 
10 quantization and variable length coding such as runlength coding, Huffman 
coding and arithmetic coding. On the other hand, JPEG2000 comprises 
discrete wavelet transform (DWT), scalar quantization, combined bit plane 
and arithmetic coding, and optimal rate control. Rate control or rate allocation 
is an algorithm or strategy to control the bit-rate of the signal coding such that 
15 it meets the target bandwidth, end-to-end delay and/or storage requirement. 
The ultimate target of rate control is to allocate the target bit-rate in the 
encoding of the signal such that the overall distortion can be minimized. In 
JPEG, the bit-rate is controlled by a single global value of quantization factor 
(or quality factor). As a result, the bit rate control is not accurate and the visual 
20 quality may vary from one region of the image to another. By using the bit- 
plane coding, JPEG2000 can control the bit-rate to meet the bit-rate 
requirement precisely and easily. And the bit rate is controlled locally and thus 
can be adapted to the local image characteristics. 

The basic encoding algorithm of JPEG2000 is based on Embedded Block 
25 Coding with Optimized Truncation or EBCOT [D. Taubman, "High 
Performance Scalable Image Compression with EBCOT," IEEE Trans, on 
Image Processing, vol. 9, no. 7, July 2000.]. The EBCOT algorithm partitions 
the wavelet coefficient into non-overlapped rectangle blocks called code- 
blocks. The code-block data are then entropy encoded by bit-plane coding. A 
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rate-distortion optimization (optimal bit allocation) process is applied after all 
the quantized wavelet coefficients have been entropy encoded (compressed) 
and is referred to as post-compression rate-distortion (PCRD) optimization [D. 
Taubman, "High Performance Scalable Image Compression with EBCOT," 
5 IEEE Trans, on Image Processing, vol. 9, no. 7, July 2000: H. Everett, 
"Generalized Lagrange Multiplier Method for Solving Problems of Optimum 
Allocation of Resources," Oper. Res., vol. 11, pp. 399-417, 1963.]. By utilizing 
the actual rate-distortion functions of all compressed data, the PCRD 
technique achieves minimum image distortion for any given bit-rate. However, 

10 since it requires the encoding of all the data and the storage of all the 
encoded bit-stream even though a large portion of the data needs not to be 
sent out, most of the computation and memory usage could be redundant in 
this process. Also the PCRD is an off-line process such that the whole image 
needs to be completely encoded before sending out any data and hence long 

15 delay is possible. 

Another technique for the optimal rate allocation of JPEG2000 is by 
coefficients modelling. Kasner et al. [J. H. Kasner, M. W. Marcellin and B. R. 
Hunt, "Universal Trellis Coded Quantization," IEEE Trans, on Image 
Processing, vol. 8, no. 12, pp. 1677-1687, Dec 1999.] assumed that the 

20 wavelet coefficients could be modelled by memory-less generalized-Gaussian 
density (GGD). By estimating the GGD parameter, the rate-distortion function 
can be approximated as required for the optimal rate allocation. This 
approach is included in Part-2 of JPEG2000 [iSO/iEC, ISO/IEC 15444-2: 
Information technology — JPEG 2000 image coding system — Part 2: 

25 Extensions, 2000.] and is called Lagrangian rate allocation (LRA). In this 
approach, both the rate and distortion are estimated before actually encoding 
the wavelet coefficients. A quantization step-size of each sub-band is selected 
based on the estimation and the quantized wavelet coefficients are encoded 
without any truncation. This approach does not have the issue of redundant 
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computation cost and redundant memory usage. However the rate control 
accuracy is heavily depended on the coefficients following the assumption of 
GGD. An iterative technique is often required to converge on the target bit- 
rate. In each iteration, the quantization step-sizes are required to be re- 
5 estimated and the wavelet coefficients are thus quantized and entropy 
encoded again. The multiple quantization and entropy encoding processes 
heavily increase the complexity of this approach. In practice, the complexity of 
LRA is comparable to the PCRD approach. 

Other than the empirical PCRD approach and the analytical LRA approach, 
10 Masuzaki et. al. [T. Masuzaki, et. al., "JPEG2000 Adaptive Rate Control for 
Embedded Systems," Proc. IEEE Int. Sym. on Circuits and Systems, vol. 4, 
pp. 333-336, May 2002.] first proposed a non-optimal training-image based 
fast rate control method for JPEG2000. By training a set of test images using 
the PCRD method, the proposed fast method obtains the relationship 
15 between the number of coding passes (coding points) and the corresponding 
number of bytes within a sub-band. The relationship is then approximated by 
a linear curve. Given a target bit-rate, the fast method can predict the number 
of coding passes to be included in the final output using the linear model. 
However the results of the paper show that this method can suffer from a 
20 significant PSNR loss (> 1dB in 0.25bpp). The loss could be much more 
significant as a single liner function cannot well approximate different kind of 
images. 

Model based rate allocation is an attractive approach for fast rate control as it 
can provide the optimal quality when the coefficients follow the model 
25 assumption. However the major drawback is the degree of model accuracy. It 
is unlikely that an accurate model can be found for highly varied images. Thus 
we change our thought into non-model based fast rate control method. 
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JPEG2000, as noted previously, is the new international standard for still 
image coding. JPEG2000 is based on the discrete wavelet transform (DWT), 
scalar quantization, coefficient bit modelling, arithmetic coding and rate 
control. The DWT decomposes an image (or sub-image called tile) into sub- 
5 bands for with different level of decomposition. Fig. 1 shows an example of 
two-level DWT decomposition. The sub-bands consist of coefficients that 
represent the horizontal and vertical spatial frequency characteristics of the 
image/tile. Each sub-band is then quantized by a scalar quantizer and divided 
into non-overlapped rectangular blocks (called code-blocks in JPEG2000) 
10 with size typically 64x64 or larger. The quantized code-block data are entropy 
encoded (compressed) to form a code-block bit-stream. Each of the code- 
block bit-stream can be truncated to meet the target bit-rate by rate control 
and finally output to the channel in packet format. 

After transformation, the wavelet coefficients are quantized using scalar 
15 quantization. Each of the coefficients a b (x,y) of the sub-band b is quantized 

to the value q b (x,y) by 



\<*b( x >y)\ 



(D 



where A^ is the quantization step size. 

In lossless compression, the value of A b must be one for all sub-bands. 

20 However, in lossy compression, no particular selection of the quantization 
step size is required in the standard. One effective way in selecting the 
quantization step size is to scale a default (or pre-defined) step size A^ by an 

energy weight parameter y 5 [J. W. Woods, J. Naveen, "A Filter Based Bit 

Allocation Scheme for Subband Compression of HDTV," IEEE Trans, on 
25 Image Processing, vol. 1 , no. 3, pp. 436-440, July 1992.] by 
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This selection of quantization step size is recommended in the standard and 
is implemented in the standard reference software [M. D. Adams and F. 
Kossentini, "JasPer: A Software-based JPEG-2000 Codec Implementation," 
5 Proc. IEEE Int. Conf. On Image Processing, vol. 2, pp. 53-56, Oct. 2000: M. 
D. Adams, "JasPer project home page," 

http://www.ece.uvic.ca/-mdadams/iasper , 2000.] with the default step size A d 
equal to two for all sub-bands. 

The quantized wavelet coefficients in the code-blocks are encoded using 

10 coefficient bit modelling and arithmetic coding. This process is called tier-1 
coding in JPEG2000. Tier-1 coding is essentially a bit-plane coding technique 
that is commonly used in wavelet based image coders [J. M. Shapiro, 
"Embedded Image Coding using Zerotrees of Wavelet Coefficients," IEEE 
Trans, on Signal Processing, vol. 41, no. 12, pp. 3445-3462, Dec. 1993: A. 

15 Said, W. A. Pearlman, "A New, Fast, and Efficient Image Codec Based on Set 
Partitioning in Hierarchical Trees," IEEE Trans, on Circuits and Systems for 
Video Tech., vol. 6, no. 3, pp. 243-250, June 1996.]. In tier-1 coding, code- 
blocks are encoded independently of one another using exactly the same 
coding algorithm. For each code-block, coefficients are encoded starting from 

20 the most significant bit-plane (MSB) with a non-zero element towards the least 
significant bit-plane (LSB). Each coefficient bit in a bit-plane is selected to be 
included in only one of the three coding passes called significance pass, 
refinement pass and cleanup pass by using coefficient bit modelling. The 
coding pass data are then arithmetic encoded by a context-based adaptive 

25 binary arithmetic coder called MQ coder in JPEG2000. 

Rate control in JPEG2000 is achieved partly by the quantization and partly by 
the selection of the coding pass data to be included in the final output (code- 
stream). The quantization process as mentioned before roughly controls the 
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rate that is generally far from the target bit-rate and is applied only once. The 
accurate rate control is achieved by selecting part of the coding pass data to 
be included in the final code-stream. JPEG2000 has no requirement on which 
rate control method to be used. However an optimal rate control process 
5 called post-compression rate-distortion (PCRD) optimization is recommended 
in the standard. This process had been described in D. Taubman, "High 
Performance Scalable Image Compression with EBCOT," IEEE Trans, on 
Image Processing, vol. 9, no. 7, July 2000 clearly and we will summarize it as 
follow. 

10 Let {#/}/ = i 5 2,... denote the set of all the code-blocks that cover the whole 

image/tile. For each code-block, an embedded bit-stream is formed by the 
tier-1 coding with a set of allowable truncation points each of which is located 
at the end of each coding pass. Thus there is at most three truncation points 
for each bit-plane. For any code block B i , the bit-stream can be truncated 

15 into different discrete length with bit-rate r),R^,... . The corresponding 
distortion incurred by reconstructing those truncated bit-streams is denoted by 

d* 1 at truncation point n t = 1,2,.... The optimal rate control process is to select 

the truncation points which minimize the overall reconstructed image 
distortion D where 

20 J> = 5>?' (3) 

i 

subject to the rate constraint 

* = <R budget (4) 

i 

where R budget denotes the target bit-rate. 
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Using the Lagrange multiplier technique [D. Taubman, "High Performance 
Scalable Image Compression with EBCOT," IEEE Trans, on Image 
Processing, vol. 9, no. 7, July 2000: H. Everett, "Generalized Lagrange 
Multiplier Method for Solving Problems of Optimum Allocation of Resources," 
5 Oper. Res., vol. 11, pp. 399-417, 1963.], the optimization process is 
equivalent to minimize the cost function 

J = D + AR = Z(*>? iW + Mt? W ) (5) 
i 

Therefore if we can find a value of x such that the set of truncation points 
which minimizes (5) and the maximum achievable rate satisfies the 

10 rate constraint in (4) will be the optimal truncation points for a target bit-rate. 

A simple algorithm in finding the optimal truncation points is mentioned in 
Taubman. At any truncation point n i% the R-D "slope" is given by 

. ^ tt : It ; — 1 it; 

S? - D >\ (6, 

* « • If • its — 1 

AH*' R* 1 -R? 

In the rest of the paper, the term R-D slope is always referred to Eqn. (6). 
15 Assume N t be the set of available truncation points for code-block B r The 

truncation point n g (A) for a given value of X is found such that 

n t (A) = max{y e N t \ sj > X} (7) 

where y = l,2 5 ... is the truncation point index. However this equation is only 

true when the R-D slope is monotonically decreasing <s" 1 ). Thus the 

20 monotonically decreasing property is assumed in the optimization algorithm. 
Based on the monotonically decreasing property, the optimal value of X 
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denoted as X optimai is equal to the minimum value of X which satisfies the 

rate constraint in (4). In practice, an iterative approach with fast convergence 
is often used in searching the X optimal . Once the X optimal is found, the optimal 

truncation points can be found by (7) with X = X optimal . 

5 In the PCRD algorithm, the R-D slope information of all the available 
truncation points are required to be pre-computed and stored in memory. This 
requires tier-1 encoding of all the quantized coefficients and the whole 
encoded bit-stream must be stored in memory even though a large portion of 
them will not be included in the final output after the optimal truncation. 

10 Therefore a significant portion of computational power and working memory 
size is wasted on computing and storing the unused data. We call this portion 
of computational power and working memory size to be redundant 
computational cost and redundant memory usage respectively. Also the 
PCRD method is a non-causal or off-line process because the entire 

15 image/tile needs to be completely encoded before sending out any data and 
hence long transmission delay is possible. Since the PCRD method requires 
tier-1 encoding of all the quantized coefficients, the computational complexity 
can be about 40% to 60% of the total CPU execution time [M. D. Adams and 
F. Kossentini, "JasPer: A Software-based JPEG-2000 Codec 

20 Implementation," Proc. IEEE Int. Conf. On Image Processing, vol. 2, pp. 53- 
56, Oct. 2000: K. F. Chen, C. J. Lian, H. H. Chen and L. G. Chen, "Analysis 
and Architecture Design of EBCOT for JPEG-2000," Proa IEEE Int. Sym. Of 
Circuits and Systems, vol. 2, pp. 765-768, May 2001 .]. 

25 Summary of the present invention 

According to broad aspects of the present invention there are provided three 
novel fast rate control methods that can efficiently reduce or remove the 
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computation and memory usage redundancy over the conventional PCRD 
method. The first method, called successive bit-plane rate allocation (SBRA), 
assigns the maximum allowable bit-rate for each bit-plane of each code-block 
by using the currently available rate-distortion information only. The second 
5 method is called priority scanning rate allocation (PSRA). This first predicts 
the order of magnitude of each truncation point's rate-distortion slope and 
then encodes the truncation points based on the order (priority) information. 
The third method uses PSRA to obtain a significantly smaller amount of data 
than PCRD for optimal truncation and is called priority scanning with optimal 
10 truncation (PSOT). 

In a comparison of the three methods, SBRA provides the highest 
computational complexity and memory usage reduction, and the lowest 
coding/transmission delay. The computational complexity reduction can be up 
to about 90% of the entropy coding process. However this method gives the 

15 lowest PSNR performance of the three. PSRA provides higher PSNR 
performance than SBRA with the penalty of lower memory usage reduction 
and higher delay. PSOT provides the best (optimal) quality while it is the least 
efficient method in term of computational complexity, memory usage and the 
coding/transmission delay. The three methods provide different degree of 

20 computation complexity and memory reduction, coding/transmission delay 
and PSNR performance. The most suitable rate control method can be 
chosen based on application requirements. 

Brief description of the figures 

Some embodiments of the invention will now be described for the sake of 
25 example only with reference to the following figures, in which: 

Fig. 1 illustrates an example of a two-level digital wavelet decomposition into 
subbands, 
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Fig. 2. shows a plot of R-D ratio curve of first code-block for test image 'Lena', 

Fig. 3 shows the indexing of subbands. Each subband is indexed by a 
resolution level and an orientation (r,0). This example shows a three level 
DWT decomposition, 

5 Fig. 4 shows selected test images, 

Fig. 5 shows the average rate difference between PCRD and SBRA-1 
(PCRD-SBRA-1 ) for sixteen subbands at different target bit-rates, 

Fig. 6 shows the scan order of (a) subband and (b) code block within a 
subband, and 

10 Fig. 7 shows the average PSNR difference between PCRD and SBRA-w 
(PCRD-SBRA-w) for different values of A at different target bit-rates. 

Detailed description of preferred embodiments 

The present invention seeks to provide new and useful rate control techniques 
for JPEG2000 image coding in particular and other forms of data coding in 
15 general. 

A first aspect of the present invention ("successive bit-plane rate allocation" or 
"SBRA") proposes that, in the determination of whether or not to include a 
coding pass in the final code-stream, a rate-distortion value (R-D value) is 
computed and compared with an adaptive threshold, if ihe R-D ratio is larger 
20 than or equal to a threshold, the coding pass would be included. This is a 
causal method which can make the decision immediately, without waiting for 
future code-blocks to be processed or encoded. 

PCRD incurs significant encoding delay because it selects the truncation 
points only after the complete tier-1 encoding of all the code-blocks. For a 
25 real-time, low delay rate control process, it is desirable that the compressed 
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data of the early code-blocks can be sent out before the later code-blocks are 
being encoded. The ideal way to do this is to select the truncation point of a 
code-block in a casual manner, in parallel with the tier-1 coding. This can also 
eliminate the redundant computation and memory usage. However, without 
knowing the actual R-D slopes of the data, it is impossible to find the optimal 
truncation point. So, instead of finding the optimal truncation point, we 
propose to find a good truncation point which is close to the optimal point in 
parallel with the tier-1 coding. 

Recall that the optimal truncation point for any code-block B i is found by Eqn. 

(7) with X = ^optimal - Ec l n - ( 7 ) requires that the R-D slope at all allowable 

truncation points to be pre-computed. However, in a causal rate control 
process, code-blocks are encoded one after another. When the code-block 
Bj is being encoded, all the R-D slope information of the future code-blocks 

{Bj}j>i is unknown. Thus the A optimal cannot be determined. 

Instead of finding the an approximation approach will be used here in 

order to obtain a causal rate control process which can be done in parallel 
with the tier-1 coding. We note that the condition sf >l from Eqn. (7) and 
becomes 




(8) 




11,-1 




'optimal 




(9) 



71,-1 



(10) 
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Now the conditions,"'' >A optimal becomes a" ! ~ ] >al' in the optimal case and 

it 1 

the term A t 1 is called R-D ratio. There are two approximations in Eqn. (10). 
The first approximation is that the R-D slope at a current truncation point n i is 
approximated by the R-D ratio of previous truncation point n t -1 . The second 
5 approximation is that the constant A optimal is now approximated by a variable 

a"* for different code-blocks and truncation points. As we do not know the 

^optimal > the optimal value of a" 1 cannot be found too. However we are not 

going to find the optimal truncation points. A good value of a"* can be simply 
found by using the rate constraint in Eqn. (4). 

10 Let the f h code-block B i be the code-block currently being encoded. Then 
{B j} - <f is the set of previously encoded code-blocks with corresponding 
truncation point {« y }y</ and {#/}y>,- is the set of future code-blocks. We 
denote {Bj}j <i and {Bj}^ as B c and B u respectively. For the current 
code-block B t , we further assume that the coefficients at and before an 

15 allowable truncation point c have been encoded such that at the final 

truncation point will satisfy n i > c . This implies R** > Rf . We then define 

^remain anc ' ^remain as 

D remain =° C i + 2>* (11) 

and 

20 R remain = R budget ~ R i ~ Z S 0 (12) 

kzB c 
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Using Eqn. (12), the rate constraint in Eqn. (4) can be rewritten as 

*,"'-*?+ ZK^Xremain (13) 
keB u 

By summing over current and all uncoded code-blocks, the condition in Eqn. 
(1 0) becomes 

where is the rate that the bit-stream is totally truncated. So its value is 

zero. As the R-D ratio A**~ l is used to approximate the R-D slope S** in 

Eqn. (10), we can assume that the term D* 1 ' /Ar"' in Eqn. (9) is constant in 
the sense that all the code-block bit-streams are truncated at the same R-D 
ratio. This also assumes that a"' is constant. The equation in Eqn. (14) then 
becomes 

+ (*,"*-*?+ Z< k ) d5) 



D remain - a \ ' R remain (1 6) 



a"' < Dremain (17) 



Using Eqn. (11) and Eqn. (13), it can be shown that a { { is bounded by 

^remain ^ ^remain ■ For D remain j ' R remain and Ec 1 n - ( 17 ) can be re-Written to 

an unbounded form as 
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g"' = Dremain ~, V^ ni >l (18) 

^remain fi^ x 

where /J. n ' is a weighting factor. As the D remain l R remain will change for 

different truncation points, a" { is eventually required to be updated for each 
truncation point. Finally we have found the equation in calculating the value of 
5 or"' and the condition in Eqn. (10) becomes 

A" 1 ' 1 =^—> remain — (19) 

Before actually encoding the sample data in-between the truncation points n i 

and 11, -1, the only unknown is Jff 1 " in Eqn. (19). Thus a maximum allowable 
rate different for truncation point n i can be computed such that 

1 0 max A*"'' = fi ? l - D* i X • Remain (2Q) 

1 1 1 D 

remain 

Tier-1 coding of code-block B g will be stopped when 

AR*' >maxA/?"'" (21) 



In other words, the bit-stream of code-block ff f . is truncated at truncation point 
n t when the condition in Eqn. (21) is satisfied. 

15 It should be noted that this approximation approach has assumed that the R- 

D ratio is monotonically decreasing (A* i+l < A?* ). However, in real situations, 

the R-D ratios evaluated at coding pass boundaries do not always follow the 
monotonically decreasing property. Fig. 2 shows the real R-D ratio curve of 
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the first code-block for test image 'Lena*. In order to have a better 
approximation, the R-D ratios may be evaluated at bit-plane boundaries 
because the distortion different between two consecutive bit-planes is larger 
than that between two consecutive coding passes. Thus the R-D ratios 
5 evaluated at bit-plane boundaries can follow the monotonically decreasing 
property much often than those evaluated at coding passes boundaries. In the 
standard, the bit-plane boundaries are defined at n t =1,4,7,10,... . Let 

b t =1,2,3,... denotes the bit-plane truncation points for code-block B t such that 

b i = 3 T W / /3~|-2. Then the stopping criterion in Eqn. (21) becomes 

1 0 - Rp > max ARp (23) 
where 

max ARp = ft ■ Dp • , Pi = J*' " 1 bi < 3 (24) 

D remain l b i ~ 3 otherwise 

Eqn. (23) and Eqn. (24) will be used in this embodiment and can be referred 
to as successive bit-plane rate allocation (SBRA). A simple form of the SBRA 

15 method is to ignore the weighting factors fi { [ such that they are all set to one. 

This non-weighted method is called SBRA-1. However different weighting 
factors other than one can be used. This weighted method may be called 
SBRA-w. 

As the SBRA method requires only the currently available R-D information 
20 and information relating to the previously encoded data can be forgotten, this 
method can provide both memoryless and casual features. The encoded 
code-block data can be sent out immediately. 
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A. Linear Rate Regulation 

The term /? bi in Eqn. (24) is used to reduce or correct the rate allocation error 

between the PCRD and SBRA method. The y? bi is defined for each bit-plane j 

and code-block /. However it is known that the image frequency 
5 characteristics are the same with a subband. Thus a simple weighting factor 

defined for each subband is often sufficient. The term /? b ' is then re-indexed 

to be (3 X e where r is the resolution level and 0 is the orientation which are 

indexed as shown in Fig. 3. 

In this modelling method, the rate profile of each subband is first examined 
10 using both the PCRD and SBRA-1 method for the fifteen test images shown in 
Fig. 4. The rates are measured in term of bit per pixel (bpp) and the average 
rate difference between PCRD and SBRA-1 is plotted in Fig. 5. It can be seen 
that SBRA-1 allocates fewer rates at lower resolution subbands and the rate 
allocation difference is approximately linear decreasing from low to high 
15 resolution subbands. The rate allocation difference can be approximated by a 
linear curve with slope related to target bit-rate. As the weighting factor in (24) 
directly controls the maximum allowable rate for a given bit-plane, a larger 
weighting factor can be used to increase the total bit-rate allocated for a given 
code-block. In other words, it is possible to have a later truncation point than 
20 that in SBRA-1. It may then be assume that the weighting factor is 
proportional to the achieved bit-rate for a given code-block. Thus one can 
model p rfi by a linear equation such that 

Prfi = l + ™* k r,# (25) 

and 
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jM-. for = (0,0) 

[M-3*(r-l)-0-l otherwise 

where m is the slope of the linear curve. M is the total number of subbands 
and (r t 0) is indexed as shown in Fig. 3. This weighting model is called linear 
rate regulation. Here it is assumed that the linear model is applied for the 
5 coding order as shown in Fig. 6. 

In general, m is dependent on image characteristics and target bit-rate. 
However a close form relationship between the m and image characteristics 
may require a complicated image analysis process. Thus model m as 

™ = *budget'& (27) 

10 where A is a pre-defined constant and R budget is the target bit-rate in term of 

bpp. The constant A is chosen by an image-training method and the PSNR 
performance is the main criterion in finding the value of A. As a five-level 
DWT decomposition is used in all simulations/implementation, the value of A 
for that configuration may be found. Fig. 7 shows the relationship between the 
15 values of A and average PSNR performance of SBRA-w using the test 
images as shown in Fig. 4. This shows that the difference of the PSNR 
performance is less than 0.1 dB in-between the values of three and ten and 
the value of five gives a relatively good result among them. Thus the value of 
a as five may be used in this embodiment. 

20 B. Implementation 

It should be noted that the coding could be stopped when maxAl**'' in Eqn. 

(24) is small enough before actually encoding the coding pass. In other words, 
it is possible to early terminate the tier-1 coding process of a code block when 
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maxAtff'' <T (28) 

In practical implementations, maxA/?*' is preferably rounded to nearest 

integer and the threshold T is set to zero because this is the simplest value 
that gives almost the highest PSNR performance. 

5 A second aspect of the invention ("Priority Scanning Rate Allocation", or 
"PSRA") proposes to establish a priority level of each coding unit of each 
code-block. All the coding units of all code-blocks are then scanned and 
encoded according to the priority level, starting from the highest priority level 
towards the lowest. 

10 By examining the PCRD scheme, it can be seen that if the R-D slopes of all 
the allowable truncation point n t are sorted in descending order and the order 

is kept in a sorted list, the optimal rate allocation can also be achieved by 
encoding the truncation point from the top of the sorted list according to the 
sorted order. This leads to an interesting observation that, once the order is 
15 known, it is not necessary to know the actual value of the R-D slope. The 
problem now becomes how to obtain the order without knowing all the values 
of the R-D slopes. 

It is unlikely that the actual order list can be obtained without knowing the 
actual R-D information. Instead the order may be predicted based on an 

20 assumption. Based on the assumed monotonically decreasing property of R-D 
slope function, an earlier truncation point will have larger R-D slope than later 
truncation point within a code-block. This property can be extended to the 
whole image such that an earlier truncation point of any code-block is 
assumed to have a larger R-D slope than later truncation points of other code- 

25 block(s). 

By this assumption, the coding priority (order) is set based on the truncation 
point level within an image. The coding pass with the higher priority will be 
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tier-1 encoded first followed by coding passes with lower priority. The 
proposed method is called priority scanning rate allocation (PSRA) and is 
performed as follows: 

1 . Initialization 

5 For the f h code block B i , the initial code-block priority is equal to the 

maximum number of coding passes required to fully encode the data in the 
code-block and is calculated by 

P, =3*^-2 (29) 

where jf is the maximum number of bit-plane level required to fully encode 
10 the data in the code-block. The current coding priority is defined as 

P^maxCPi) (30) 
i 

2. Priority Scanning 

Visit each code-block according to the scan order as shown in Fig. 6. For 
each code-block B i , condition C1 is checked. 

15 C1 : If the code-block priority is equal to the current coding priority (P { = P c ), 

encode the first unencoded coding pass of B i , reduce Pj by one, 

calculate the R-D slope by Eqn. 4 and check the condition C2. Else 
check the condition C3. 

C2: if the accumulated bit-rate is larger than the target bit-rate, find and 
20 discard the encoded coding pass(es) which has the minimum R-D 

slope(s) such that the accumulated bit-rate is less than or equal to the 
target bit-rate. The whole tier-1 coding process will be terminated at 
this point. Else check the condition C3. 
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C3: If the code-block is the last one in the scan order, reduce^ by one and 

the next code-block will be the first one in the scan order. Else visit the 
following code block. 

The code-block priority in Eqn. (20) determines the truncation point level to be 
processed in the current round within an image. It exploits the assumption that 
the highest truncation point level has the highest R-D slope within an image. 
The current coding priority function in Eqn. (30) is just an indication to indicate 
what priority will be included in the current scanning. Based on the actual R-D 
functions, the discard of the included coding pass in the condition C2 can 
maximize the PSNR performance of this method. 

The PSNR performance of PSRA might be degraded when the coefficients do 
not follow the assumption. However experimental results show that PSRA can 
have good PSNR performance for most test images. Different from SBRA, the 
encoded code-block data in PSRA can only be sent out after the whole PSRA 
process is finished. 

A third aspect of the invention ("Priority Scanning with Optimal Truncation", or 
"PSOT") proposes establishing a priority level of each coding unit of each 
20 code-block. All the coding units of all code-blocks are then scanned and 
encoded according to the priority level, starting from the highest priority level 
towards the iowest. When sufficient coding levels of the code-blocks are 
encoded, rate-distortion optimized allocation is applied. 

The PCRD method achieves the best visual quality but is not the most 
25 efficient of the three methods because it requires encoding the whole set of 
the allowable truncation points. Compared with PCRD, PSRA encodes a 
significantly smaller set of truncation points but the visual quality is less than 
optimal. Here a new method called Priority Scanning with Optimal Truncation 
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(PSOT) is proposed and which is able to achieve optimal visual quality similar 
to PCRD, and computational complexity similar to PSRA. 

First, the PSRA method is used to estimate a R-D slope that could be close to 
the optimal x . Then a minimum slope rejection method is used to terminate 
5 the tier-1 coding process of each code block such that the last calculated R-D 
slope of each code block is less than or equal to the estimated R-D slope. 
Finally, PCRD is applied to the encoded truncation points. By doing so, 
instead of encoding all truncation points, only a smaller amount of truncation 
points are required to be encoded. This proposed scheme is called priority 
10 scanning with optimal truncation (PSOT) and is performed as follows: 

1. Initialization & Priority Scanning 

Perform PSRA as in Section IV. However, instead of termination, if the 
accumulated bit rate is larger than the target bit rate, go to step 2. 

15 2. Minimum slope rejection 

Find the minimum rate-distortion slope S min among the calculated R-D slopes 
in step 1. Visit each of the code-blocks and continue the Tier-1 encoding until 
the code block's minimum rate-distortion slope is less than or equal to the 

Smin- 

20 3. Optimal Truncation 

This step is essentially the PCRD method. The optimal X is found among the 
encoded truncation points. Then the truncation points with rate-distortion 
slope greater than or equal to the optimal iare included in the final code- 
stream. 

25 If the Smin is less than or equal to the optimal x , the proposed method will 
have the same PSNR performance as the conventional PCRD scheme in 
JPEG2000. Same as PSRA, the encoded code-block data can only be sent 
out after the whole PSOT process has been finished. 
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The embodiments of the invention described above are in terms of JPEG2000 
image coding. The code-block can be of any shape (e.g. square or 
rectangular). The blocks may or may not be overlapping. Furthermore, it is not 
necessary to follow the coding order of the code-blocks in JPEG2000, and it is 
5 not necessary to define the coding passes as the significance pass, 
refinement pass and cleanup pass of JPEG2000. Other coding passes are 
possible, including embedded zero-tree wavelet coding, or those of SPHIT, 
etc. Furthermore, it is not necessary to use discrete wavelet transform. Other 
transforms such as discrete cosine transform, discrete Fourier transform, 
10 Hadamard tranform, Slant transform, etc. are possible. 

For the images, one picture element may have one or more components such 
as the luminance component, the red, green, blue (RGB) components, the 
YUV components, the YCrCb components, the infra-red components, the X- 
ray or other components. Each component of a picture element is a number, 
15 which may be a natural number, an integer, a real number or even a complex 
number. In the case of natural numbers, they may be 12-bit, 8-bit, or any 
other bit resolution. 

Moreover, the present invention in any of its aspects is applicable not only to 
the encoding of images, but also to the rate control or rate allocation in the 
20 encoding of audio signals, speech signals, video signals, seismic signals, 
medical signals, etc. 

The fast rate control process is mainiy targeted for fast, low-delay and low 
cost software and hardware implementations of JPEG2000. Possible 
applications include digital camera, digital camcorder, personal digital 
25 assistant (PDA), multimedia-enabled cellular phones (2.5G, 3G, and beyond), 
wireless LAN devices, Bluetooth applications, web servers, internet 
applications, scanners, remote sensing, etc. In digital cameras, delay is very 
important. People often complain that, after depressing the button, the camera 
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would take a long time to capture and store the image. This is becoming an 
increasing important problem as the digital images are getting bigger and 
bigger. Nowadays, some cameras have 5 mega pixels (about 2560x1920 
pixels) or more, which would require a significant amount of time to encode. 
The proposed algorithm can reduce the time to store the image because it 
reduces significantly the time to convert the image into the JPEG2000 format 
(by about 40%). The camera can be ready to take another picture much 
sooner. In additional, our algorithm requires much lesser memory than PCRD 
so that cache memory requirement in hardware can be reduced and the 
implementation cost can be reduced. Nowadays, many digital video (DV) 
camcorders include still image capture capability. They would store the 
captured image in a separate storage media such as multimedia card, or SD, 
etc. Users would experience similar complaints as they do in digital cameras, 
namely, a long delay in storing the images resulting in long interval between 
consecutive image shots. Again the proposed algorithm can reduce the delay 
significantly. Similarly, many PDAs, cellular phones, wireless LAN and 
Bluetooth devices, and notebook computers are increasing having 
video/image capture capability built-in or in add-on devices. A long delay in 
storing images would always result in long interval between consecutive 
image shots, and the proposed algorithm can reduce the delay. 
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